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1.  Introduction  and  Project  Overview 

A  major  challenge  in  pre-hospital  field  settings  is  the  accurate  identification  of  wounds,  in  particular  burn  wounds, 
such  that  the  clinician  (e.g.  medic)  can  provide  optimal  care.  Specifically,  the  identification  of  the  bum  size, 
location,  and  depth  are  critical  to  initiating  appropriate  therapies  in  the  field.  Currently,  Joint  Trauma  System  (JTS) 
guidelines  recommend  using  the  “Rule  of  Nines”  to  estimate  the  burn  wound  size  or  to  use  a  Lund-Browser  chart  if 
available.  Recent  data  comparing  pre-burn  center  and  burn  center  wound  size  estimation  demonstrated  an  average  of 
7+7%  (range  0-32%)  difference  between  novice  and  expert  estimates.  These  differences  included  both  over- 
(novices  estimated  the  burn  size  to  be  larger  than  the  expert)  and  under-estimating  (novices  estimated  the  bum  size 
to  be  smaller  than  expert)  the  extent  of  injury.  Many  (43%)  were  over  estimations.  Rarely  was  the  novice  estimate  of 
burn  size  the  same  as  the  expert  estimation,  whereas  experts  compared  to  experts  usually  estimate  burn  size  within 
5%  of  each  other.  These  discrepancies  introduce  large  amounts  of  error  into  the  standard  equations  used  to  predict 
resuscitation  fluid  volumes  during  the  first  24  hours  post-burn  injury.  Civilian  data  is  similar:  novice  clinicians 
routinely  miscalculate  the  size  of  burn  wounds,  with  differences  between  novice  providers’  assessments  and  those  of 
experts  being  reported  to  be  as  large  as  100%.  This  variance  results  in  significant  differences  in  the  amount  of  fluid 
used  to  resuscitate  patients  and  increases  the  potential  for  resuscitation  morbidity. 

In  response  to  these  challenges,  the  USAISR  developed  and  obtained  FDA  510(k)  clearance  of  the  Burn 
Navigator™,  a  computer  decision  support  system  to  guide  fluid  resuscitation  in  the  field  environment.  The  device  is 
designed  to  assist  novice  burn  care  providers  with  resuscitation  during  the  first  24-48  hours  of  resuscitating  patients 
with  large  burn  wounds.  Unfortunately,  inaccurate  burn  wound  size  estimation,  when  entered  into  this  computer 
decision  support  software  (CDSS),  can  significantly  change  the  CDSS  algorithm’s  recommendations  and  thus  the 
total  fluid  administered  to  a  patient.  The  objective  of  this  project  is  to  develop  a  decision  support  software 
application  that  will  aid  novice  bum  care  providers  with  the  assessment  of  thermal  injuries.  The  Specific  Aims  of 
this  proposal  are  as  follows: 

1 .  Create  a  multispectral  image  database  of  burn  wounds  that  can  be  used  for  machine  learning  and  a  de- 
identified  database  of  Enhanced  Burn  Wound  Diagrams; 

2.  Create  and  teach  a  Machine  Learning  (ML)  algorithm  to  estimate  burn  severity  and  size  from  imaging  data 
to  within  5%  an  expert’s  opinion; 

3.  Produce  a  reference  design  for  a  portable  imaging  and  burn  wound  prediction  design. 

The  objectives  of  this  project  include: 

Objective  1:  Place  images  of  human  subjects  onto  a  standard  anatomic  diagram  of  the  human  body  and  create  a 
multi-spectral  image  database  of  burn  wounds  for  ML. 

Task  1.1:  Provide  a  multispectral  camera  or  other  advanced  imaging  platform  for  image  data  collection  at  the 
USAISR 

■  ARA  will  deliver  camera  to  USAISR 

■  USAISR  submits  and  obtains  IRB  and  IACUC  approvals. 

■  USAISR  images  healthy  volunteers,  burns  on  a  porcine  burn  model,  and  patients  admitted  to  the  burn 
center. 

Task  1.2:  Create  de-identified  database  of  enhanced  burn  wound  diagrams  (multi-spectral  burn  wound  images 
superimposed  on  standard,  digital,  anatomical,  burn  wound  mapping  diagram  of  the  human  body). 

■  Software  engineers  at  the  USAISR  and  OSU  create  enhanced  burn  wound  diagrams. 

■  USAISR  sends  de-identified  imaging  data  to  ARA  for  preprocessing. 

Task  1.3:  Produce  multi-spectral  burn  wound  images  and  enhanced  burn  wound  diagrams  for  automated,  multi¬ 
spectral  feature  extraction  and  image  segmentation. 

■  ARA  will  pre-process  multi-spectral  bum  wound  images  to  validate  that  registration  and 
transformation  into  the  standard,  digital,  anatomical,  burn  wound  diagram  of  the  human  body,  does  not 
introduce  disruptive  artifacts. 

Objective  2:  Compare  results  of  the  burn  wound  mapping  and  total  body  surface  area  (%TBSA)  estimates  derived 
from  clinical  experts  using  current  best  practices,  existing  technologies  and  innovative  multi -spectral  machine 
learning. 

Task  2. 1 :  Expert  annotation  of  enhanced  burn  wound  diagrams  within  the  de-identified  data  set. 

■  USAISR  will  transfer  de-identified  enhanced  data  set  (minimum  of  10)  to  ARA  and  OSU.  The  data  set 
will  include  de  identified  data  generated  by  using  a  standard,  digital,  anatomical,  burn  wound  mapping 


Page  4  of  14 


DM167028 


diagram  of  the  human  body,  depth  maps  from  the  camera  and  multiple  spectral  images  from  the 
camera. 

■  OSU  uses  enhanced  burn  wound  diagrams  and  their  existing  software  to  trace  wounds  and  estimate  of 
%  TBS  A. 

Task  2.2:  Experts  in  burn  wound  care  at  USAISR  estimate  wound  size  and  burn  depth;  review  OSU’s  enhanced  burn 
wound  diagrams  to  confirm  or  correct  these  estimates  as  needed 

■  Create  a  machine  learning  model.  (Months  5-19) 

■  ARA  uses  training  data  set  to  create  ML  algorithms  to  automatically  estimate  burn  wound  depth  and 
%TBSA. 

■  ARA  uses  the  evaluation  cohort  to  demonstrate  the  ML  algorithm  reaching  within  5%  of  expert’s 
judgment 

Ob  jective  3:  Teach  machine  learning  software  to  model  raw  images  of  burn  wounds. 

Task  3.1:  Use  the  training  data  set  from  the  multispectral  burn  wound  image  data  and  the  enhanced  bum  wound 
mapping  with  clinical  expert  annotations  to  create  ML  algorithms  to  automatically  estimate  burn  wound  depth  and 
per  cent  TBS  A  burned. 

Task  3.2:  Use  the  evaluation  data  set  to  demonstrate  the  ML  algorithm  will  determine  meaningful  patterns  identified 
in  the  spectral  data  to  predict  the  type,  size,  and  location  of  burn  wounds  within  5%  of  a  clinical  expert’s  judgment. 

Objective  4:  Produce  a  reference  design 

Task  4.1:  ARA  will  produce  a  reference  design  for  a  mobile  (hand  carried  or  helmet  mountable)  camera -computer 
platform  that  incorporates  high  definition  imagers  and  the  machine  learning  modeling  software  usable  in  a  PLC 
setting. 

2.  Technical  Progress 

a.  Team  Management 

The  ARA  team  assumed  project  responsibilities  following  the  award  of  the  contract  on  September  1,  2017.  Due  to 
negotiation  issues  and  a  mismatch  in  technology,  RVS  was  dropped  from  the  team  before  the  subcontract  to  them 
was  awarded.  ARA  acquired  and  will  be  delivering  an  alternate  camera  system  directly  to  USAISR  instead.  OSU  is 
not  yet  under  subcontract,  but  will  be  by  the  time  data  acquisition  begins. 

b.  Protocols 

There  are  three  protocols  for  this  project,  including  the  Laboratory  Protocol  to  govern  initial  data  collection  using 
manikins  and  healthy  volunteers,  an  Animal  Use  Protocol  to  govern  collection  of  porcine  burn  wound  images  for 
initial  ML  development,  and  a  Human  Use  Protocol  to  govern  collection  of  human  burn  wound  images  for  both 
training  and  validation  of  the  BURNMAN  ML  software. 

•  Laboratory  Protocol  -  We  have  written  and  submitted  a  laboratory  protocol  for  initial  software 
development  using  digital  images  of  manikins  to  the  USAISR  regulatory  department.  The  USAISR  SOP’s 
are  currently  still  under  revision,  and  the  status  of  our  protocol  is  pending  these  revisions  which  may  direct 
a  different  pathway  for  approval  for  this  type  of  work  (i.e.  preparatory  research  and  development ).  In  the 
meanwhile,  we  continued  our  efforts  to  take  images  using  the  Microsoft  Kinect  system  for  initial  software 
development  for  this  device. 

•  Human  IRB  protocol  -  Dr.  Jeremy  Pamplin  has  written  the  first  draft  of  a  human  protocol.  We  have  made 
modifications  to  the  first  draft  version  upon  our  discovery  that  the  MS  Kinect  device  may  not  be  feasible  for 
this  project.  We  have  been  unable  to  finalize  the  protocol  until  we  know  what  type  of  thermal  imaging 
camera  we  will  be  using  for  the  study.  This  information  is  still  pending.  ARA  just  came  under  contract  in 
September,  2017.  It  has  recently  been  determined  that  a  sub  contract  with  Raytheon  will  not  occur,  this  in 
conjunction  with  ARA,  we  are  in  the  process  of  market  research  with  currently  available  Commercial  off 
the  Shelf  (COTS)  cameras.  Once  we  receive  the  camera  and  learn  about  its  functionality  and  limitations, 
we  will  revise  the  human  protocol  accordingly;  send  it  out  for  scientific  review  and  for  pre-review  through 
the  USAISR  regulatory  department. 

•  Animal  protocol  -  We  started  collaborations  with,  Dr.  Randolph  Stone,  PhD,  who  routinely  conducts 
several  IACUC  approved  porcine  burn-wound  research  studies.  Dr.  Stone  has  experience  developing  swine 
burn  wound  models  and  taking  pictures  using  different  types  of  thermal  imaging  and  multi- spectral 
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imaging  cameras.  Taking  wound  images  is  a  non-interventional  study,  so  protocol  amendments  did  not 
need  to  be  submitted  per  our  IACUC  regulations.  Ms.  Veazey  has  been  added  as  an  Associate  Investigator 
on  Dr.  Stone ’s  current  protocols.  Dr.  Stone  has  agreed  to  allow  Mr.  F enrich  to  take  photos  and  did  not 
need  to  be  added  to  the  protocols.  Ms.  Veazey  and  Mr.  Fenrich  have  completed  the  required  animal 
training.  Ms.  Veazey  and  Mr  Fenrich  has  fulfilled  the  regulatory  and  occupational  health  requirements  to 
conduct  research  in  our  vivarium.  Ms.  Serio-Melvin  still  has  a  few  classes  to  complete.  Our  studies  have 
begun  in  September. 

c.  Equipment  Acquisition 

We  have  acquired  and  evaluated  multiple  cameras  for  potential  use  in  this  project.  These  include  the  Kinect, 
RealSense,  FLIR  One,  Hero  5  GoPro,  and  Euclid  cameras.  A  short  review  of  each  is  provided  below.  With  the 
dropping  of  RVS  and  the  fact  that  the  original  Pixnet  camera  is  not  available  for  this  project,  we  have  elected  to  use 
the  MSI  Camera  for  multispectral  burn  image  data  acquisition.  Various  cameras  were  evaluated  to  ensure  we 
identified  the  deal  methodology  of  detecting  a  human  body  while  removing  the  background. 


Kinect:  The  IR  capabilities  of  the  Kinect  system  are  only  used  for  depth 
detection.  The  Kinect  system  has  a  clipping  plane  of  1  meter  to  8 
meters;  therefore  any  object  closer  than  1  meter  or  further  from  8 
meters  will  be  clipped  from  view.  The  IR  image  resolution  is  also 
smaller  (at  512x424  pixels)  while  the  visual  image  resolution  is  larger 
(at  1920x1080  pixels).  All  body  mapping  work  performed  to  date  was 
completed  using  the  Kinect  camera  and  is  described  in  the  sections  below. 

Intel®  RealSense™  Camera  ZR3 00-Series  does  human  body  detection 
and  uses  an  infrared  camera  similar  to  the  Kinect  system.  The  IR 
camera  of  the  RealSense  Camera  is  used  for  depth  detection  as  is  the 
IR  camera  of  the  Kinect.  This  camera  also  utilizes  the  OpenCV  library 
to  assist  in  the  body  detection  algorithm  that  is  support  with  the  SDK  that  comes 
with  the  camera. 

FLIR  One  Pro  camera  connects  to  an  Android  smart  phone/device  and  can  take 
thermal  imagining  pictures  while  connected  to  the  smart  phone.  After  evaluation, 
we  determined  that  the  FLIR  One  did  not  have  sufficient  resolution  to  work  for  our 
application. 

Intel  Euclid  Camera  includes  the  RealSense™  depth  camera  technology,  a 
motion  camera,  and  an  Intel®  Atom™  x7-Z8700  Quad  core  CPU  to  produce  a 
compact  and  sleek  all-in-one  computer  and  depth  camera.  This  systme  includes 
the  same  hardware  as  the  RealSense  on  its  own,  but  with  an  integrated  CPU. 

Hero  5  GoPro  camera  is  a  small  camera  and  can  be  controlled  by  an  App  that 
you  download.  You  can  connect  to  it  via  Bluetooth  or  Wifi  but  the  distance  in 
which  you  can  control  the  camera  is  not  very  long.  The  camera  did  not  appear  to 
provide  any  additional  use  to  the  requirement  for  pictures  usable  in  body  mapping. 
SpectroCam  VIS-NIR  5MPixel  camera  system.  We  selected  the  Spectrocam  based 
on  a  thorough  market  survey  of  current  COTS  technology  and  after  review  of  the 
technology  offered  by  RVS.  This  system  offers  the  necessary  resolution, 
flexibility,  and  spectral  bands  to  support  the  research  objectives. 


A  mounting  system  has  been  acquired  for  the  cameras  which  attaches  to 
a  standard  Computer  on  Wheels  (COW)  cart  in  the  Burn  Center.  When 
acquiring  animal  images,  the  camera  system  will  be  mounted  on  a 
similar  cart  used  in  the  vivarium. 
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d.  Image  Capture  Method  Development 

Here  we  describe  the  processes  taken  to  identify  the  ideal  methodology  of  detecting  a  human  body  while  removing 
the  background.  Our  first  objective  is  to  develop  software  capabilities  while  leveraging  existing  tools,  such  as  the 
Kinect  SDK  and  available  libraries  for  various  programming  languages,  to  scan  a  body  and  remove  background 
artifacts.  The  figures  below  were  taken  by  our  lead  biomedical  software  developer  at  the  USAISR,  Craig  Fenrich, 
and  with  permission  are  used  to  describe  our  results  for  this  quarterly  report  to  show  the  proof  of  concept. 

i.  3D  Object  Scanning 

3D  Object  scanning  is  necessary  to  capture  a  patient  and  register  burn  wound  images  to  specific  body  regions  for 
application  on  a  standard  burn  wound  diagram.  We  initially  planned  to  use  the  Kinect  Infrared  camera  for  this  task 
and  initial  work  was  performed  with  this  system.  Our  first  approach  was  to  scan  a  human  body  as  a  3D  image 
because  the  Microsoft  Kinect  SDK  provides  a  3D  depth  map  that  can  be  translated  into  a  3D  image.  We  believed 
that  this  method  would  be  the  most  accurate  in  assuring  that  the  person  scanned  along  with  the  bums  would  have  the 
least  amount  of  digital  manipulation  to  accurately  depict  a  burn  on  a  patient  body  as  opposed  to  a  2D  representation 
(A  3D  body  provides  more  accurate  a  body  surface  area  estimation  than  a  2D  representation). 

When  we  initially  processed  the  3D  mapping,  the  program  required  the  subject  to  be  relatively  stationary.  For  a  full 
body  scan,  the  sensor  (Kinect)  must  be  moved  slowly  and  in  a  linear  fashion  otherwise  it  would  distort  the  3D 
image.  Additionally,  the  results  showed  that  the  overlay  of  the  3D  image  with  texturizing  in  relation  to  IR  distance 
produced  an  image  that  included  all  objects  within  the  clipping  plane  (seen  in  Figure  1). 

To  develop  utilizing  the  Kinect  SDK  along  with  the 
integrated  development  environment,  Visual  Studio  2017, 
we  had  to  code  in  C#  for  the  depth  map.  However,  to 
incorporate  both  depth  imaging  and  3D  image  development, 
a  3D  graphics  library  such  as  OpenGL  for  C#  was  required. 

We  looked  for  the  3D  Open  Graphics  Library  (Open  GL) 
for  C#  but  the  version  available  is  not  currently  supported; 
however  we  did  find  a  Java  library  for  the  Kinect  system 
(J4K)  that  uses  OpenGL  for  3D  and  uses  a  port  of  the 
Kinect  SDK  to  Java. 

We  used  a  Java  based  library  called  J4K  library  along  with 
OpenGL  and  incorporated  3D  development,  but  the 
processing  speed  was  extremely  slow.  Potentially,  multiple 
Kinect  sensors  could  be  used  to  increase  the  processing 
speed  to  simultaneously  scan  a  subject  at  the  same  time  but 
is  not  applicable  for  this  project. 

Based  on  our  results,  we  then  proceeded  with  2D 
development  and  continued  with  the  J4K  Java  libraries.  We 
created  a  depth  map  along  with  edge  map  detection  using 
IR.  The  edge  image  that  was  created  mapped  the  picture  of 
the  body  using  the  body  frame  from  the  Kinect  SDK  or  the  distance  mapping.  Although  this  method  worked,  the 
resolution  of  the  depth  image  from  the  IR  and  visual  image  did  not  match  correctly  using  the  current  Java 
capabilities.  Therefore,  we  implemented  the  original  Kinect  SDK  with  C#  methods  which  had  the  capabilities  for 
correlating  the  IR  depth  map  (512X424)  bitmap  to  the  visual  picture  (1920X1080)  bitmap.  After  testing,  the 
decision  was  made  to  implement  C#  instead  of  Java. 

Below  in  Figure  2,  on  the  right  side  of  the  figure  in  red,  the  IR  depth  map  image  displays  the  infrared  depth  map. 
Based  on  the  shading,  the  color  gradient  displays  the  distance  from  closest  (brighter)  to  further  (darker).  Depth  and 
visual  overlay  pictures  are  shown  in  Figure  3  and  4.  The  Kinect  also  has  an  edge  detection  function,  shown  in  Figure 
5. 


sitting  still.  The  texture  and  depth  map  shows 
incomplete  depiction  of  the  subject  and  the 
surrounding  environment. 
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Figure  3  Depth  image  with  visual  overlay 
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Figure  5  Edge  detection  image  (right) 


ii.  Body  Mapping 

We  have  tested  3  different  methods  for  mapping  the  body  apart  from  the  background  or  any  other  artifact.  Based  on 
our  previous  results,  we  have  eliminated  the  3D  body  mapping  because  it  produced  more  errors  in  body  recognition 
and  was  considerably  slower  in  processing.  The  three  methods  we  have  identified  and  are  currently  testing  are  1) 
Body  frame  detection;  2)  Distance  reference;  and  3)  Chroma  key  technique. 

1.  Body  Frame  Skeletal  Detection  from  Kinect  SDK 

The  Kinect  system  incorporates  a  machine  learning  algorithm  for  body  tracking.  It  is  able  to  track  up  to  25  joints  of 
the  body.  Therefore,  the  body  tracking  allows  the  program  to  discern  not  only  a  body  frame  from  the  environment 
but  also  the  movable  limbs.  This  has  been  useful  for  detecting  specific  movements.  Based  on  our  evaluation,  the 
sensitivity  and  accuracy  of  the  body  tracking  is  not  as  stable  and  can  jitter  when  the  program  is  evaluating  the  body 
joints  in  real  time.  Below  in  Figure  6,  the  joint- skeletal  frame  is  tracked  in  real  time. 
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Figure  6  Body  Tracking  Image 


2.  Distance  Reference  from  Kinect 

The  distance  reference  method  works  by  telling  the  program  the  distance  of  where  the  subject  is  compared  to  the  rest 
of  the  background.  Example,  the  subject  must  be  within  8  meters  of  the  camera  and  everything  beyond  it  is 
considered  background.  Based  on  our  initial  results,  we  found  that  this  method  is  less  accurate  because  it  will  also 
incorporate  any  other  object  within  that  distance  as  well.  Example  seen  in  Figure  7,  the  arms  of  the  chair  are  in  the 
same  plane,  therefore  are  included  in  the  distance  reference  picture. 


Figure  7  Distance  image 


3.  Chroma  Key 
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The  third  method  utilizes  a  Chroma  (or  color)  key  that  uses  a  color  as  a  reference  point  for  the  background.  This 
method  is  also  known  as  the  “green  screen”  that  is  used  in  video  when  adding  or  removing  an  artificial  background. 
We  used  an  open  source  Chroma  Key  algorithm  and  modified  it  to  work  with  the  camera  on  the  Kinect.  We  took 
photos  using  the  Kinect  camera  and  ran  the  picture  through  the  Chroma  Key  algorithm. 

Using  Chroma  key,  we  can  utilize  any  dark  contrast  color.  Here  in  Figure  8,  we  used  blue  sheets  and  towels  that  are 
commonly  used  in  the  Burn  Intensive  Care  Unit  (BICU)  as  the  Chroma  key.  The  program  detected  the  color  and  has 
colored  it  in  pink.  Because  the  shirt  is  also  blue,  it  was  also  added  to  the  Chroma  key. 


Figure  8  Body  Tracking  and  Chroma  key  Image 


Using  the  Chroma  key  methodology  our  next  step  was  to  test  the  MS  Kinect  with  a  mannequin  lying  on  the  floor  to 
simulate  a  burned  combat  casualty.  We  discovered  that  the  body  frame  (skeletal)  detection  from  Kinect  did  not 
perform  correctly  (or  at  all)  when  the  Kinect  was  pointed  downward.  Upon  further  investigation  we  found  that  it  was 
the  accelerometer  built  into  the  Kinect  system  that  disables  its  function  unless  pointed  forward  at  a  person  standing 
or  sitting.  Since  this  was  not  acceptable  for  what  we  were  trying  to  accomplish  we  decided  to  look  at  using  a 
Chroma  Key  methodology  for  body  frame  detection. 

Believing  that  the  Chroma  key  methodology  had  some  potential,  we  focused  our  efforts  on  refining  our  Chroma  key 
algorithm.  This  algorithm  utilizes  a  blue  background  and  replaces  the  blue  background  with  the  human  body 
diagram  from  the  Tactical  Combat  Casualty  Care  (TC3)  card.  (See  figure  9.)  We  placed  the  mannequin  on  blue 
sheets  on  the  floor  and  took  pictures  using  the  Kinect  to  test  this  methodology.  This  appeared  to  be  working  as 
shown  in  the  photos  below. 
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Figure  .10  Picture  of  mannequin  lying  on  floor  using  Chroma  Key 

Since  the  Chroma  key  methodology  would  only  utilize  the  RGB  camera  of  the 
Kinect  system  and  not  take  advantage  of  the  IR  camera  of  the  Kinect  which 
produces  the  depth  map,  and  would  also  require  a  separate  computer  connected  to 
the  Kinect  (not  feasible  for  the  field  environment)  we  developed  an  Android 
application  that  would  allow  the  Chroma  key  algorithm  to  utilize  the  camera  on  a 
smart  phone.  This  is  shown  below  in  Figure  10. 

We  also  searched  for  an  Open  Source  SDK  that  could  do  body  silhouette  detection 
as  does  the  Kinect  but  would  work  when  the  camera  was  pointing  in  any  direction 
and  found  an  SDK  called  OpenCV.  The  proof  of  concept  of  this  SDK  worked,  but 
the  OpenCV  library  does  not  do  actual  body  silhouette  detection  like  the  Kinect  is 
capable  of  doing;  it  does  more  of  human  detection. 

iii.  Image  Stitching 

Image  stitching  is  necessary  to  connect  multiple  pictures  of  the  same  patient 
into  a  single  registered  image.  If  multiple  photos  of  burn  wounds  are  take  on  a 
single  patient,  these  images  must  be  stitched  to  a  common  frame  of  reference, 
the  burn  wound  image  diagram.  We  tested  two  open  source  computer  vision 
libraries,  one  was  OpenCV  and  the  other  was  BoofCV  to  support  image  stitching. 

We  first  took  photos  of  a  mannequin  in  the  BICU  with  the  hope  of  being  able  to  stitch  together  the  images  of  all  of 
the  body  parts,  we  also  used  a  blue  background  to  also  test  the  ChromaKey  which  is  also  part  of  the  OpenCV 
library.  Unfortunately,  the  mannequin  has  very  few  distinctive  points  for  the  stitching  algorithm  in  either  library  to 
be  able  to  stitch  the  images  together  so  a  fiduciary  marker  in  the  form  of  a  QRF  code  was  manually  inserted  on  a 
photo  of  the  trunk  and  the  left  arm.  We  ran  the  algorithm  again  using  both  BoofCV  and  OpenCV.  OpenCV  did 
slightly  better  at  stitching  the  photos  together. 


Figure  9  Picture  taken  with 
Chroma  key  application 
and  an  Android  smart 
phone 
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Figure  1 1  Mannequin  arm  with  QRF  marker,  Image  12  Mannequin  trunk  with 
marker,  Image  13  images  stitched  together  using  OpenCV. 

We  took  numerous  photos  of  a  human  volunteer  using  four  sheets  with  the  QRF  marker  placed  at  the  left  shoulder, 
right  shoulder,  left  hip,  and  right  hip.  However,  based  on  this  brief  experiment,  we  determined  that  a  unique  marker 
is  necessary  for  each  location.  The  next  round  will  be  done  with  four  unique  markers. 


Figure  12  Body  Silhouette  using  Canny  Edge  Detection 

3.  Animal  Burn  Wound  Image  Capture 

The  following  are  active  animal  protocols  during  which  we  may  obtain  images.  No  amendment  or  new  protocol  is 
necessary  for  passive  image  capture  during  these  ongoing  animal  protocols. 

•  Protocol  #A- 16-021 -TP:  Evaluation  of  Tissue  Engineered  Therapies  in  a  Porcine  (Sus  scrofa)  Skin  Injury 
Model  -  A  Type  Protocol  Animal  Protocol  PI:  Randolph  Stone  II,  PhD 

•  Protocol  #A1730  :  Optimization  of  Debridement  Depth  that  Results  in  Complete  Graft  Take  for  Porcine 
Burn  Wounds  Animal  Protocol  PI:  Randolph  Stone  II,  PhD 

o  Protocol  A1730:  First  images  were  taken  of  Porcine  experiment  #9901  on  Sept  7,  2017.  Porcine 
model  had  approximately  2X2"  (xl2)  squares  not  including  tattoo  area  with  each  square  with 
varying  burning  times  to  capitulate  varying  burn  severity  (0  seconds  -  15  seconds).  Histology 
report  of  actual  burn  depth  has  not  been  recorded  at  this  time.  We  took  images  of  approximately  2 
hours  post  burn  using  the  FLIR  ONE  pro  (Android  device)  camera, 
o  Protocol  A1730:  We  took  images  of  a  pre-burn  porcine  model  on  Sept  8,  2017.  Porcine  had  12 
tattooed  squares  but  with  no  burns;  we  wanted  to  test  out  the  capabilities  of  the  FLIR  ONE  camera 
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and  develop  a  workflow  for  taking  visual  and  IR  images  as  well  as  documentation  and  distance 
from  skin  to  camera.  We  also  realized  we  needed  a  reliable  thermometer  and  humidity  device  as 
additional  data  points. 


4.  Deliverables  Status 

The  deliverables  for  the  BURNMAN  project  to  date  are: 

1.  Multispectral  Camera  Delivery:  Nov  24,  2017 

2.  Animal  Use  Protocol  amendment  approval:  NA. 
only  per  USAISR  IACUC. 

Pending  deliverables  include: 

1.  Human  Use  Protocol:  Draft  protocol  written. 

The  following  are  planned  for  the  next  reporting  period: 


B 


a 

o 


No  animal  protocol  needed  for  passive  image  capture 


a.  Installation  of  multispectral  camera  system  in  the  Animal  ICU  for  burn  wound  image  capture 

b.  Animal  burn  wound  data  for  ML  model  development 

c.  Enhanced  burn  wound  diagram 

The  figure  below  shows  the  updated  project  research  schedule  in  Gantt  chart  format.  It  has  been  adjusted  from  the 
original  submission  to  account  for  the  development  delays  associated  with  ARA’s  contract  award. 
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2016 

2017 

2018 

2019 

Task  Numbe 

Task  Description 

Jun 

Jul 

Aug 

Sep 

Oct 

Nov 

Dec 

Jan 
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Feb 

Mar 

Apr 

May 

Jun 

Jul 

Aug 

Sep 

Oct 

Nov 

Dec 

Jan 

Feb 

Mar 

Apr 

May 

Jun 

Jul 

Aug 

Sep 

Objective  2 

Create  Dataset 

Task  1 

Deliver  Multispectral  Camera 

The  USAISR  submits  and  obtains  IRB  and  IACUC 
approvals 

Capture  Manikin  images  for  processing 

Capture  animal  burn  wound  images 

Capture  human  burn  wound  images 

Task  2 

Create  enhanced  burn  wound  diagrams 

Task  3 

Pre-process  multi-spectral  burn  wound  imaaes 

Objective  3 

Compare  results  of  the  burn  wound  mapping  and 
total  body  surface  area  (%TBSA)  estimates 

Task  1 

Receive  de-identified  enhanced  data  set  from 
USAISR. 

Task  2 

Use  enhanced  burn  wound  diagrams  and  their 
proprietary  software  to  trace  wounds  to  estimate 
percentage  (%)  of  TBSA 

Task  3 

Send  enhanced  burn  wound  diagrams  estimated 
percentage  (%)  of  TBSA  to  USAISR  for  clinical 
experts  review 

Task  4 

Receive  de-identified  enhanced  data  sets  with 
USAISR  clinical  expert  annotations,  split  into 
traininq  and  evaluation  sets 

Objective  4 

Create  a  Machine  Learning  (ML)  software  model 
to  create  a  predictive  model  of  burn 

Task  1 

Create  ML  algorithms  to  automatically  estimate 
burn  wound  depth  and  per  cent  TBSA  burned. 

Task  2 

Demonstrate  the  ML  algorithm 

Task  3 

Compare  ML  output  to  clinical  expert  assessment 
of  burn  wounds 

Objective  5 

Produce  the  reference  design  for  a  mobile  camera- 
computer  platform 

5.  Administrative 

Applied  Research  Associates,  Inc.  (ARA)  has  been  under  Contract  W81XWH-12-C-0126  to  the  U.S.  Army  Medical 
Research  &  Material  Command’s  (USAMRMC)  Telemedicine  &  Advanced  Technology  Research  Center  (TATRC) 
for  two  months.  We  have  ordered  the  spectral  camera  with  anticipated  delivery  of  Nov  24. 

Meetings  -  The  team  participates  in  regularly  scheduled  team  meetings  and  occasional  WebEx  conferences  to 
further  review  and  discuss  the  development  of  BURNMAN. 

•  BURNMAN  bi-weekly  project  technical  progress  meetings  (Thursday) 

•  BURNMAN  bi-weekly  project  management  meetings  (Friday) 

6.  Equipment  and  Supplies 

We  have  ordered  the  Multispectral  camera  for  the  project,  a  Kinect  camera  and  Dell  Precision  7510  laptop  for 
managing  the  cameras. 

The  following  supplies  have  been  ordered  for  the  project: 

•  Extendable  camera  boom 

•  Hardware  for  mounting  the  camera  boom 

•  2-camera  mounting  platform 

•  Intel  RealSense  Camera 

•  Microsoft  Kinect  Camera 

7.  Reportable  Outcomes 

There  are  no  reportable  outcomes  to  date  for  this  project. 

8.  Conclusions 

The  BURNMAN  project  has  only  just  started  with  the  award  of  the  contract  to  ARA  and  OSU.  The  first  year  was 
spent  primarily  in  coordination  activities  and  some  initial  technology  review  to  identify  the  best  technologies  for 
image  data  acquisition. 

We  have  made  reasonable  progress  in  technology  review,  identifying  a  number  of  candidate  cameras  and  down- 
selecting  to  the  best  for  use  in  the  project.  Now  that  the  technology  has  been  identified  and  ordered,  data  acquisition 
will  begin  in  earnest  during  the  next  reporting  period. 
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